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ABSTRACT 

A study determined whether participation in a project 
using personal computers in the classroom to teach the writing 
process enabled students to improve their writing performance. One 
middle school from each of Delaware's 16 districts participated in 
the program. The project goals were incorporated into a set of four 
process and three outcome evaluation questions which formed the 
framework for the evaluation design. Results indicated that: (1) 
adequate orientation and training were provided to the schools; (2) 
most program sites started on time and implemented the program as 
planned; (3) a few of the schools did not take advantage of the 
special training opportunities and technical services offered; (4) 
on-site evaluation and monitoring of the first year was essential; 
(5) students' writing skills were enhanced through the use of 
computers within the context of the process approach to writing; (6) 
students enjoyed writing more when using computers in conjunction 
with their normal writing instruction program; (7) studer*-s enjoyed 
using the computer for learning and practicing their writing skills; 
(8) teachers enjoyed using the computer for writing instruction; (9) 
variables fostering continued implementation were support from the 
school district, teachers' increased knowledge and comfort, and 
student interest in and support of the practice; and (10) variables 
mitigating against the program were the limited number of computers, 
lack of school or district support, large class size, and lack of 
student interest. (Six tables of data and a figure describing the 
evaluation design are included.) (RS) 
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Introductio n 

The definition of literacy, which is commonly agreed to be a major 
educational goal for our students, has changed. It is no longer merely the 
acquisition of the basic skills of reading and writing, but now includes the 
thoughtful use of skills such as reading, writing, speaking, and listening 
(Brown, 1987). Calfee (1992) describes this new focus as the development of 
critical literacy, n the capacity to use language in all its forms as a tool 
for thinking, problem solving, and communicat ing* (p. 148). Whatever 
definition one adopts, this educational goal is not being met. 

One of the most depressing accounts of America's faii'ire to develop 
literate abilities in our students is the results of NAEP's 1^84 and 1988 
national assessments ot representat ive samples of fourth eighth, and 
eleventh grade students' performance on various writing tasks (Applebee, 
Langer, & Mullis, 1988; Applebee, anger, Mullis, & Jenkins, 1990). These 
studies found that most of the 18,000 students sampled knew some or all of 
the basic elements needed to complete the writing tasks but were unable to 
provide "adequate" responses that reflected using the elements to 
effectively accomplish the tasks* Also, thete was little improvement in 
writing levels across the four year period. Along with tneir poor 
pe: t onnance , most students did not report having positive attitudes toward 
writing. Another salient finding was a trend for teacheis to use 
proees s-ol iented instructional activities, although whether or not students 
used these inst : uct ional strategies effectively, frequently, and with 
understanding was unclear. However, the NAEP results did suggest that when 
students plan, revise, and edit they £ie more likely to be better writers. 

The Delaware Department of Public Instruction (DPI) identified a state 
need to improve student writing performance and, based on the state's 
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Additionally, it was anticipated that the project would have a positive 
impact on teachers in their classroom instruction. These three hypotheses 
guided the development of the evaluation design. 

The program had five principal components or key elements: 

• teacher daily use of the writing process approach to teaching 
writing, focusing on the stages of pte-writing, drafting, and 
revis ing 

• teacher daily use of all computers fur students, on a rotating 
basis, to work through their writing assignments 

• teacher daily use of a short writing lesson or mini- lesson, thus 
allowing students to spend most of their class time working on 
writing assignments 

• teacher daily use of conferencing, with individuals or small groups 
of students, to encourage students' writing 

• teacher on-going monitoring of students working on the computer and 
on writing assignments, to be aware of their progress and needs tor 
assistance. 

Each of the five components of the program needed to he carried out 
effectively if the desired outcomes were to he realized. 

Desi gn and Procedures 
The computer writing project was implemented in each of sixteen school 
districts throughout Delaware. Writing teachers in each of the selected 
middle schools (one per district) participated and received at least six 
eoin,uit his, one printer, and writing software. All hardware/ software 
selections were made by the teacher in conjunction with local administra- 
tors, in addition, thirteen ot the sixteen participating schools provided a 
control group for the study. These control groups were selected on the 
basis ot comparability ot teachers and students and on the expectation that 
the control students would have limited exposure to the use of the computet 
in learning writing skills. 
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There were five stages to the project, each of which was to be 
evaluated. They were: 1) orientation of participating teachers, 
2) selection and acquisition of hardware and software, 3) training of 
participating teachers, 4) implementation, and 5) continuation. The first 
three stages were completed during the spring and summer of 1989. Then, 
teachers began, early in the school year, to use the computers in their 
daily instructional activities. Throughout the 1989-1990 school year, 
teachers were monitored and were provided with technical assistance and 
support as needed. A follow-up evaluation at the end of year two, the 
1990-1991 school year, looked at the continuaticn of the sixteen original 
implement at ions , 
Evaluation Design 

The project goals were incorporated into a set of four process and 
three outcome evaluation questions which formed the framewoik for the 
evaluation design. This framework is outlined in Figure 1. 

The process component of the evaluation was to focus on insuring that 
prerequisite conditions for proper program implementation were met . It was 
therefore designed to study the course of the five stages of the project, to 
document their progress, and to assess the quality of their associated 
activities. Feedback from the process evaluation was to be used by project 
administrators and staff to adjust and fine-tune project activities in order 
to make the activities more effective. 

The outcome component ot the evaluation was designed to assess the 
effects of the project on student writing skills and on student attitudes 
coward writing and toward using the computer tor writing. A pretest- 
post test control group design was used to enhance the validity ot the 
findings. Guidelines for selecting the control or comparison Krnup hu each 
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Figure 1 
EVALUATION DESIGN 
Process Evaluation 



Evaluation Question 



Measure 



ERIC 



Orientation - Was adequate 
orientation and assistance 
provided for planning and 
selection of hardware /software? 



• Readiness Checklist 

• Teacher Inservice 
Evaluation Form 



Selection and Acquisition - Was 
the appropriate hardware and 
software ordered and delivered 
on time? 



School Hardware/Software 
Decisions Form 
Telephone Readiness 
Checklist 



Training - Was adequate inservice 
training in computer literacy, 
the writing process, and the 
use of computers in teaching 
writing provided to project 
teachers 7 

Implementation - Did the teachers 
start-up on time and implement the 
key program elements (writing 
process approach, computer use, 
mini- less ions f conf erenc ing, 
monitoring) ? 



# 



Software Training 
Evaluation Form 
Teacher Inservice 
Evaluation Form 



Telephone Readiness 
Checklist 

Classroom Observation Form 
Teachers Log 
Teacher Feedback 



Continue t ion - rid the original 
program implementations continue 
and what mitigated against/ 
f sosr ereri continuation / expans ion? 



Teacher Follow-up Survey 



Expectation 



• Corranitment and support for 
the project. 

• Ready to explore hardware/ 
software options. 

• Training needs identified 

• Necessary and appropriate 
hardware /software selected, 
ordered, and received by 
September. 

• Training needs met. 

• Implementation concerns identifed. 

• Any outstanding needs to be 
met by September 1. 



# All training, hardware, and 
software needs satisfied. 

# Program on schedule for start-up 
in September* 

# Regular use of key program 
elements in classroom by all 
teachers . 

# Continued use of key program 
elements in classroom by all 
teachers (at the same or 
increased levels) . 
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EVALUATION DESIGN (continued) 



Evaluation Question 

Writing Skills - Were student 
writing skills enhanced a? a 
result of participation in 
the project? 



Outcome Evaluation 
Measure 

• Pre- and Post- Writing 
Assessment 

• Pre- and Post- Student 
Survey 



Enjoyment of Writing - Did 
students get more enjoyment 
out of writing as a result 
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participation in the 



project? 

Enjoyment of Computer Learning 
Did students enjoy using the 
computer for learning and 
practicing writing skills? 



• Pre- and Post - Student 

Survey 
B Teacher Anecdotes 



& Pre- and Post - Student 

Survey 
• Teacher Anecdotes 



Expectation 

# Significantly greater writing 
proficiency' in program group 
over comparison group. 

e Significantly improved writing 
skills by self-report in program 
group over comparison group . 

# Significantly enhanced enjoyment 
of writing by self-report in 
program group over comparison group. 

# Teacher reports of students' enhanced 
enjoyment of writing. 

# Student responses indicate enjoyment 
in using the computer for writing 
assignment s . 

# Teacher reports of students' 
enjoyment of using the computer for 
writ ing . 



project class included specifications that the two groups he as similar as 
possible on key characteristics (i.e., type ot student, .chievement levels, 
demographic variables). Feedback from the outcome evaluation was to 
document overall program effectiveness and to p.ovide information usetui in 
making decisions about project expansion or replication. 
Procedure s 

Figure 1 describes the process and outcome evaluation design in terms 
of evaluation questions, measures, and expectations for each of the four 
stages ot the project. The evaluation procedures associated with the five 
process and three outcome evaluation questions are described below. 

Or ientat ion . The sixteen project teachers and several of their school 
administrators participated in a two-day workshop in the spring of 1989, 
The workshop was designed to provide an orientation to the project, and an 
overview of writing software, computer tuxdwait', and approaches for using 
the computer to assist with writing instruction. During the first 
orientation session, teachers were administered the Readiness Checklist to 
determine their district/school's readiness to participate in the project. 
At the conclusion of the second day of the orientation workshop, 
participants were administered the Inservice Evaluation Form to obtain 
feedback on the orientation sessions and also to assess their readiness to 
select t hf ueiessaiy hardware and software for the project. 

A separate orientation session was held tor principals and compute: 
coordinators from the sixteen middle schools. This session was designed to 
provide administrators with an overview of the project and the ioquired 
implementation tasks {e.g., m participate in hardware and software 
selection) and to address any tvm ^rns or fjuejU ions that might arise pi ior 
to implement at ion , This activity was observed and evaluated by RBS staff. 
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Selection and Acquisition, In June, a School Hardware/ Software 
Derisions Form was completed by administrators in each district in 
conjunction with the project teacher, indicating the type of computer 
hardware and software package(s) the districts decided to purchase for the 
project. As a follow-up, a Telephone Readiness Checklist was used to 
determine whether the hardware and software ordered had been received, 
set-up, tested, and was ready to he used for instruction- 

Tra ining. The Teacher Inservice Evaluation Form served is a vehicle 
for identifying teacher need tor further training on computer hardware, 
software, or on the writing process. To respond to these identified needs, 
two August workshops were held; the DPI conducted a fifteen hour course on 
the Delaware Writing Process for which teachers received one inservice 
credit, and fifteen hours of training on the specific hardware and software 
teachers selected was provided by DPI staff, RBS staff, and representatives 
from computer vendors. At the conclusion of the workshops, RBS administered 
a Software Training Evaluation Form to assess the effectiveness of the 
training and to identify additional needs, it any. 

Impleme nta tion. The implementation of the project was addressed by the 
proce s and outcome evaluation components. The process evaluation focused 
on monitoring teacher use of the five program elements (i.e., the writing 
process, the computers, min i- lessons , conferencing, and monitoring), 
information was collected through three process evaluation activities, 
classroom observations, teacher logs, and follow-up meet ings . Initial 
ini oiinat ion on stait-up dates foi implement ing the program was obtained by 
telephone calls to principals vi the participating schools early in the 
school year. 
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A Classroom Observation Form was developed to collect information 
during observations of participating teacher classrooms. Three rounds of 
observations were conducted, at the beginning, the middle, and the end of 
the school year. Teachers' self-report data on their implementation of the 
program elements was documented through a Teacher Log and through 
discussions at follow-up meetings. Three follow-up dinner meetings were 
convened during year one, and another meeting was held during year two. The 
meetings were designed to: 1) facilitate sharing/communication among project 
participants, 2) identify any additional assistance and support needed by 
participants to implement the project, and 3) develop plans to provide the 
needed assistance in a timely fashion. 

Continuation. Due to the limited resources and time for conducting the 
follow-up evaluation of the computer writing project, the primary data 
source to address the continuation and possible expansion of the project was 
feedback from a teacher survey. The survey was to be administered to 
project teachers during a group meeting at the end of the 1990-1991 school 
year and would address impl ^ment ation of the five program components, 
expansion within and outside of project classes, and factors supporting or 
constraining continuation and expansion. 

The outcome evaluation focused on the impact of the project on student 
writing proficiency, and on student enjoyment of writing as well as using 
the computer for writing. As indicated previously, the evaluation plan 
incorporated a pret est-posttest control group design. 

^;:At in B S kills . in older to assess student writ ing skills, project and 
control teachers administered a Pie-Writing Assessment at the beginning of 
the school year, and a Post -Writing Assessment at the end of the school 
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year. The assessments required students to produce a short writing sample 
in response to the following prowipts: 

• "Think about one change that you would like to make in your school 
and why you would like to make that change. Give reasons why the 
change should be made, and explain how the change would benefit the 
school..." (Pre-Writing Assessment). 

• "Think about one change that you would like to make in the town or 
city in which you live and why you would like to make that change. 
Give reasons why the change should be made, and explain how the 
change would benefit your town or city..." (Post-Writing 
Assessment ) . 

These prompts were similar to those used in the 1985 Delaware Writing 
Assessment Project, and were determined to have the potential for producing 
a range of scorable student responses. They were also judged to be free of 
bias, interesting to students, and unambiguous. 

Enjoyment of Wilting and Computer Learning. Information on student 
attitudes was collected through a Student Survey attached to each writing 
assessment. Each survey consisted of the same eleven multiple choice items 
addressing student experiences with writing assignments and computer use. 
Additional impact data were obtained through teacher reports and anecdotes 
shared at follow-up meetings and during school visits and telephone 
contacts . 

Re sults 

The following section of the report presents the results of the data 
analysis. The finding? are discussed in terms of the process and outcome 
evaluation questions. 
Process Evaluation Results 

The results of the process evaluation activities are reported with 
reference to the five process evaluation questions outlined in Figure 1. 

Was adequate orientation and assistance provided for planning and 
select ion of ha rdware/soft ware? Thirteen project t earners attended the 
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two-day orientation session. The Readiness Checklist administered during 
the first day indicated that teacher's perceptions of the district/school 
leadership's commitment and support for the project, as well as the 
teachers' own commitment toward the project, were very high. The results 
also indicated that, while teachers had previously attended workshops on the 
writing process, they had only limited experience using a computer in 
personal writing tasks. 

The results of the Teacher Inservice Evaluation Form, administered at 
the conclusion of the second day of the orientation workshop, were very 
positive. In terms of implementation readiness, respondents indicated an 
awareness of the writing process and how to use the computer to teach 
writing for classroom instruction. Although most teachers were exploring 
hardware and software options with administrators, a few indicated that 
either the selections were being made for them or that they were not able to 
contribute to the decisions. Most teachers also indicated a need for 
further training on the hardware, software, and to a lesser extent on 
the writing process. To address these training needs, a week-long workshop 
was scheduled for the end of August. 

RBS f s evaluation of the orientation session tor principals and computer 
coordinators indicated that the objectives oi the session wer*> met. In 
addition, participants exhibited enthusiasm and support for the project. 

Was the appropriate hardware and software ordered and delivered on 
Ljjggj The School Hardware/Software Decisions Form, completed by 
administrators in each district in conjunction with the project teacher, 
indicated that all sixteen schools selected and purchased at least six 
computers, one printer, and writing software by early June. In terms of the 
computer hardware, eight districts selected Apple Macintosh computers (Plus, 
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SE, II), five selected Apple He computers, one district selected Smart 
Micro (IBM compatible), one selected Apple IIGS, and one selected Hyundai. 
The writing software selected and ordered to support implementation of the 
project included: Writer's Helper II (seven districts), Success with Writing 
(two districts), and Bank Street Writer III (two districts). Also, each of 
the following word processing software package?; was selected by two 
districts? MacWrite , Word , Wordbench , WordPerfect , and Works > 

The results of the Telephone Readiness Checklist show only two 
principals indicating that their project teachers definitely needed 
additional hardware and software training; all principals felt that training 
on the writing process was sufficient. In most cases, the hardware and 
software had been received and set-up, and only one principal reported that 
the equipment had not been tested, A few principals, however, reported 
minor problems (e.g., all materials software, printers, discs not 
received, electrical hook-ups and computer tables needed, defective 
monitor). These problems were followed-up by RBS and the DPI. 

Was adequate inservice training in computer literacy, the wri ting 
process, and the use of computers in teaching writing provided to project 
teachers? Nine project teachers participated in the workshop on the 
hardware and selected software. Feedback on the Software Training 
Evaluation Form showed that high ratings were given to both the sessions and 
the presenters. Overall, teachers felt that they received the support they 
needed for using the hardware and software. In terms of implementation 
readiness, they reported ieeling comfortable with using the software tor the 
writing and revising stages, but a little less comfortable with using the 
software fur prewriting or planning. Although most teachers felt that they 
needed additional practice, their major concern seemed to be with managing a 
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classroom which contained computers. These concerns were monitored by the 
DPI and RBS through school visits and personal contacts. 

n id the teachers start-up on time and impl ement the key program 
elements? Program start-up dates were provided by principals as part of the 
Telephone Readiness Checklist, administered in late August. Eight 
principals indicated the start-up date for implementing the program would be 
in September, and eight indicated the program would start "as soon as 
possible." The DPI and RBS continued to monitor each teacher's progress in 
beginning implementation. Although most teachers began using their 
computers by October, the initial start-up generally involved several weeks 
of "trouble-shooting" (i.e., setting-up the classroom, debugging software, 
managing the classroom, ordering materials, maintaining Teacher Logs). 

Information on program implementation was collected through conducting 
classroom observations, using the Classroom Observation Form, reviewing 
Teacher Logs, and discussing the project with teachers aid students. 
Although the Logs were scanned during observations and requested for 
submission at the end of the school year, only half of the project teachers 
recorded this information on a regular basis; the remaining teachers 
reported it was difficult to find time to recall and enter th required 
information in the typical "hectic" school day. However, those who did 
maintain the Teacher Log indicated that it provided useful information for 
their instructional planning. 

Over the course of the first round of observations, technical problems 
were resolved and almost totally eliminated. When a school was identified 
as needing technical support and assistance, this information was fed-back 
to the DPI and follow-up was provided. For example, computer vendors were 
contacted if hardware was the problem, the DPI provided technical assistance 
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to help implement software and address software problems, and visits among 
project staff were facilitated. 

In terms of teaching writing, these observations indicated that all 
teachers were focusing their instruction on the writing process and a few 
teachers were using a mini-lesson format. However, in discussions with 
teachers, most indicated that they did use mini-lessons on a regular basis, 
but use of such lessons varied with the class and assignment. Also, 
approximately half of the teachers were observed conferencing with 
individual students. Again, it was not expected that all students would be 
observed to be at this stage in the writing process. In several classes 
students were observed conferencing with each other, an activity which some 
teachers were using prior to or in lieu of student- teacher conferencing. 
This strategy seemed to work well; students appeared to enjoy the 
"author-to-author* interaction with their peers and to have the necessary 
skills to provide helpful feedback. Overall, teachers were observed 
providing extensive individual monitoring and assistance to students working 
on their writing assignments and also to students working on computers* 

Observat ions concerning computer use were also very positive. Teachers 
had developed a system for scheduling all students to be on the computer one 
or more times a week, usually to work on and complete at least one assign- 
ment. In many classes the schedule was posted. Teachers had from six to 
ten computers available (and 1 or 2 printers), and they were all in use dur- 
ing each observation, by one or two students, for an average of 30 minutes. 

Teachers were observed trying out two differer* instructional 
strategies for incorporating computer use into their English classes. Some 
had all students working on the same writing assignment, with some students 
completing the assignment on the computer. Others had groups of students 
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and RBS recommendations for continuing the project; and, RBS summarized the 
observation findings and discussed the evaluation and documentation plans. 
Several teachers even shared their students 1 writing and special class 
writing projects- Teachers informally reported these dinner meetings to be 
very helpful and welcomed the opportunity to meet and share information with 
other project teachers. 

Did the original program implementations continue and what mitigated 
against /fostered continuation/expansion? The follow-up survey was completed 
by 14 of the 16 project teachers; six completed the survey during the May 
1991 meeting and eight teachers completed the survey by mail- All 
respondents indicated that their students continued to use the computer, on 
a rotating basis, to work on writing assignments, and in addition to 
drafting and editing, six teachers reported that students* computer work 
also focused on pre-writing. All respondents also indicated that they 
continued to use on a daily basis the writing process approach, and thirteen 
respondents reported that they continued to use mini-lessons and to monitor 
students* computer use on a regular basis- Twelve of the teachers 
responding reported that they continued to use conferencing on a routine 
basis with individuals or small groups of students. With few exceptions, 
the reported levels of implementation of these five program elements were 
equal to i>r greater than those at the previous year. Interest ingly , nine 
teachers reported that they were much more comfortable using the computers 
for writing instruction during the 1990-1991 school year. 

Teachers were asked about expansion within and outside nf project 
classes. The hardware and sottware used by project teachers werp basically 
the same as that used in yea* one, even in cases where the district provided 
funds for purchasing additional software. However, four respondents 
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indicated that the program spread to other classes within their school- In 
one case, the program spread was to other teachers in the same building. In 
a second case, the expansion was a district rather than a project effort. A 
third teacher indicated there was minimal spread to other teachers (due to 
the limited number of computers). The fourth case was expansion as a result 
of the project teacher presenting an elective computer writing course. 

The following factors were most often cited by teachers as supporting 
the continuation of their computer writing programs during the 1990-1991 
school year* support from school or district in terms of hardware, software, 
supplies, equipment maintenance, and training; teachers' increased knowledge 
and comfort; and student interest in and support of the project. Factors 
reported as mitigating against continued implementation and expansion were: 
the limited number of computers; lack of school or district support for 
hardware, supplies, maintenance, and training; large class size and 
shortened length of class period; and lack of student interest in and 
support of the practice. 
Outcome Evaluation Results 

Three outcome evaluation questions were specified in the design of the 
study as described earlier. The findings with respect to each of these are 
presented below. 

Were student writing skills enhanced as a result of participation in 
the project? Student writing skills were assessed for experimental and 
control students on a pre- and posttest basis. A] 1 pre- and post-writing 
assessments were sent to an independent third party for scoring (CTB 
MacMillan/McGraw-Hill) . A total of 5,887 papers were scored. The scoring 
system involved four analytic factors: focus, organization, development, 
and language conventions. The focus factor was used to assess the clarity 



and point or subject and its appropriateness to the audience and purpose as 
specified by the prompt* The organization factor dealt with the clarity and 
effectiveness of the plan or arrangement of ideas and the use of 
organizational methods or strategies appropriate to the audience and purpose 
of the prompt. The development factor involved the elaboration of the main 
point or subject using examples, specific details, and supporting 
information. The conventions factor was used to assess the correct use of 
standard English including grammar, sentence structure, spelling, 
punctuation, and capitalization. 

The factors were scored on a six-point scale according to separate sets 
of criteria and recorded as four independent scores plus an equally weighted 
total score. The six-point scale ranged as follows: (1) seriously 
deficient, (2) moderately deficient, (3) slightly deficient, (4) moderately 
proficient, (5) proficient, and (6) exceptionally proficient. Specific- 
criteria lor review of each factor are presented in the appendix. All 
papers were scored by two independent readers, and discrepancies {papers 
with score differences of more than one point) were scored a third time 
(adjudicated) by table leaders. Practice sets and validation packs were 
developed and used for training of scorers. Scoring procedures employed 
initial training, practice sample scoring, validation procedures and 
consistency checks. Reader reliability checks were made and reports issued 
beginning with the second day of scoring. Condition codes were recorded in 
place of scores when one of the following response conditions were 
encountered during scoring: (A) blank, (B) illegible, (C) off topic, (D) 
insufficient to score, or (E) predominately in another language. 

A final score was arrived at for each of the four factors, and a total 
score consisting of the sum of the final four factor scores was calculated. 
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In obtaining these scores, certain guidelines were followed. If there were 
only two readings, the final score was simply the average of the two reading 
scores. If a third reading was necessary , the final sore was the third 
reading score. If the same condition code was given for the two reading 
scores, the condition code was used as the final score. However, if two 
different condition codes were given or a numerical code and a condition 
code were given, the final score was the third reading score. The total 
score was computed as the sum of the four factor scores, with condition 
codes assigned a value of zero. 

Two approaches were used in the analysis of ".he data on student writing 
skills. The first involved an analysis of covariance of mean writing 
posttest scores comparing experimental and control group students across all 
schools. Table 1 shows the results of this analysis. The second approach 
analyzed program effect: by comparing gains for the experimental and the 
corresponding control group within each school. The results of this latter 
approach are shown in Tables 2-6. In comparing gains for the experimental 
and control group within each school, schools not having a control group 
were evaluated by using the overall control group school mean in place of 
what would have been the school's control group. 

Table 1 shows results of an analysis of covariance of posttest writing 
assessment scores using the pretest as a covariate. Separate analyses were 
conducted tor each score. Results show the total number of subjects 
analyzed <N), the adjusted posttest means for experimental (X) and control 
CC) groups, the F value (F), and the level of statistical significance 
obtained {Sig.). 

As can be seen in Table 1, the results reflect very favorably on the 
experimental program Results of comparisons between experimental and 
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Table 1 



Analysis of Covariance of Post Student Writing Assessment 
(using Pretest as the Covariate) 



Adjusted Fustte st 
Mean 



Store 


N 


X 


C 




F 


Sig. 


Total 


2285 


16. 


11 


15. 


82 


9. 


34 


.002 


Focus 


2275 


4. 


,18 


4. 


15 


1. 


, 78 


NS 


Oi ganizat ion 


2285 


3 , 


.93 


3. 


83 


13. 


.13 


.001 


Development 


2285 


3. 


,97 


3. 


90 


4. 


, 79 


.02 


Convent ions 


2285 


4. 


. 03 


3. 


9b 


8 , 


. 89 


.003 



X - ExjjHi inieniai 
C = Cont tol 
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Table 2 

Writing Assessment Gains By School 
(Total Score) 



School 


Group * 


N 


Pre 


Post 


Gain 


t-sig 


Gr. Gain 


[A] 


X 


84 


15.80 


16.07 


.27 


NS 


X 


C 


158 


16.10 


15.72 


- .38 


.05 





[B] 


A 


47 


15.17 


15.91 


.74 


.01 


p 


tr 






.04 


NS 


[C] 


X 


74 


13.44 


14.47 


1.03 


.001 


r 
t 


oi 


1 J . CO 


1 J .O/ 


CO 




[D] 


X 


81 


14.14 


15.70 


1.56 


.001 


r 
L 


901 


1 A AA 


1 A dA 
ID • HO 


1 A 7 


no. 1 


[E] 


X 


89 


13.54 


17.18 


3.64 


.001 


r 


fiO 






1 «H J 




[F] 


X 


82 


13.41 


14.79 


1.38 


.001 




r 

L 


Ol 


1 J * :JO 


Id AA 


AA 
.00 


• UUl 


[G] 


X 


108 


17.05 


18.01 


.96 


.001 




c 




ID »UD 


1 A Aft 


A 7 

- .4/ 


NS 


[H] 


X 


107 


13.36 


14.74 


1.38 


.001 




r 
t 




1 J • lO 


1 c 07 

11) .5/ 


cn 




[I] 


X 


49 


17.71 


18.98 


1.2/ 


.001 




p 

c 


38 


15. 91 


16.20 


.29 


NS 


[J] 


X 


22 


14.68 


14.41 


- .27 


NS 




c 


83 


15.28 


15.87 


.59 


— mm M , 


M 


X 


89 


15.00 


14.80 


- .20 


NS 




p 


/in 


1 A fit 


16 .20 


1 .25 


.01 


£L] 


X 


58 


16.45 


16.89 


.44 


NS 




c 


97 


15.04 


16.30 


1.26 


.001 


[M] 


X 


67 


16.78 


16.87 


.08 


NS 




c 


35 


15.60 


16.11 


.51 


NS 


[N] 


X 


98 


14.96 


15.91 


.95 


.001 




c 


17 


15.15 


15.56 


.41 


NS 


[0] 


X 


67 


13.44 


14.75 


1.31 


.001 




c 


141 


15.58 


15.73 


.15 


NS 


M 


X 


59 


15.09 


17.06 


1.97 


.001 




c 


107 


13.60 


14.29 


.69 


.01 


MEANS 


X 


74 


15.00 


16.03 


1.03 






c 


83 


15.28 


15.87 


.59 





X 
X 

c 

X 
X 
X 
X 
X 

c 
c 
c 
c 

X 



11=X 
5=C 



X= Experimental 
C=Control 
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Table 3 

Writing Assessment Gains By School 



(Focus Score) 



School 




N 


Pre 


Post 


Gain 


t-siq 


Gr. Gain 


[A] 


X 


84 


4.12 


4.21 


.09 


NS 


X 


r 


I JO 


4 19 


4 12 


- .07 


NS 




[B] 


X 


47 


4.04 


4.11 


.07 


NS 


X 


r 




4 74 


4 28 


.04 


NS 




[C] 


X 




3.58 


3.80 




.005 


X 


r 








18 






[D] 


X 


81 


3.76 


4.12 


.36 


.001 


c 


r 


cu 1 




4 29 


.39 


.001 




[E] 


X 


88 


3.59 


4.44 


.85 


.001 


X 


r 




4 05 


4 .50 


.45 


.001 




[F] 


X 


82 


3.55 


3.85 


.30 


.001 


X 


r 


O i 


3 71 

•J * f A 


3.91 


.20 


.001 




[G] 


X 


108 


4.41 


4.65 


.24 


.001 


X 


r 


14 


4 ?fi 


4 11 


15 


NS 




[M] 


X 


107 


3.63 


3.86 


.23 


.001 


X 










18 






[I] 


X 


49 


4.61 


4.86 


.25 


.05 


X 




JO 




4 1 1 


03 


NS 






X 


22 


3.77 


3.79 


.02 


NS 


c 










18 






[KJ 


X 


89 


3.92 


3.85 


- .07 


NS 


c 


r 


4Q 


1 R9 


4 27 


33 


01 




[L] 


X 


57 




4.44 


.05 


NS 


c 


r 






4 2Q 


34 


001 

■ \J\J A 




CM] 


X 


67 


4.40 


4.44 


.04 


NS 


c 


r 




4 rn 


4 1ft 


n 


NS 






A 


Q7 


Oft 




1 1 


NS 


/• 
V. 


C 


17 


3.94 


4.09 


. i r > 


n\ 




[Oj 


X 


67 


3.63 


3.89 


.;•{» 




X 




c 


141 


4.07 




.07 


NS 




[P] 


X 


58 


3.98 


4.40 


.42 


.001 


X 


c 


105 


3.62 


3.74 


.1? 


NS 




MEANS 

X = Kxpo 


X 

c 


74 
83 




27 


~22 
.18 




10-X 
6=C 



q C - Control 
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Table 4 

Writing Assessment Gains By School 



(Organization Score) 



School 


* 

Group 


N 


Pre 


Post 


Gain 


t-siq 

..--it 


Gr. Gain 


[A] 


X 


84 


3.84 


3.98 


.14 


NS 


X 


r 




3 90 


3 81 


- 09 


NS 




[B] 


X 


47 


3.59 


3.85 


.26 


.005 


X 


r 




«J t OU 


3 95 


09 


NS 

no 




[C] 


X 


74 


3.28 


3.49 


.21 


.005 


X 




f 








17 

• if 






[D] 


X 


81 


3 36 


3.86 


.50 


.001 


X 


c 


201 


3 59 


3 97 


38 


001 




[E] 


X 


89 


3.22 


4.27 


1.05 


.001 


X 


c 


60 


3.82 


4.20 


.38 


.001 




[FJ 


X 


82 


3.26 


3.59 


.33 


.001 


X 




r 


ftl 




1 5fi 

«J * JO 






[G] 


X 


108 


4.16 


4.41 


.25 


.001 


X 




c 


36 


3.93 


3.79 


.14 


NS 




CM] 


X 


107 


3.18 


3.59 


.41 


.001 


X 




c 








17 






[I] 


X 


49 


4.29 


4.60 


.31 


.01 


X 




r 


38 


3 87 


3 88 


01 


NS 




[J] 


X 


22 


3.56 


3.45 


- .11 


NS 


c 




r 








17 






[K] 


X 


89 


3.63 


3.53 


- .10 


NS 


c 




r 


4Q 








» Ul 




[L] 


X 


58 


3.97 


4.14 


.17 


.05 


c 




r 


Q7 


1 fiQ 


1 Q7 








[M] 


X 


67 


4.06 


4.10 


• 04 


NS 


c 




r 




"3 77 


1 Qfl 










x 


98 


^ S7 


} Aft 

j * on 




no i 

■ WW! 


X 




C 


17 


3.65 


3.74 


.0 f J 


NS 




[0] 


X 


0/ 


J . 1 / 




. 45 


. 00 1 


X 




c 


141 


3.77 


3.79 


.0? 


NS 




CP] 


X 


59 


3.64 


4.17 


.53 


.001 


X 




c 


107 


3.26 


3.46 


.20 


.005 




MEANS X 
C 

X ~ Experimental 


74 
83 






.30 
.17 




12=X 
4=C 



C = Control >S? 
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Table 5 

Writing Assessment Gains By School 



(Development Score) 



School 


Group 


N 


Pre 


Post 


Gain 


t-sig 


Gr. Gain 


[A] 


X 


84 


3.86 


3.87 


.01 


NS 


X 


C 


158 


3.97 


3.79 


1 ft 

- .18 


.01 




[B] 


X 


47 


3.67 


3.88 


.21 


.05 


X 


c 


65 


4.14 


3.98 


- .16 


NS 




[C] 


X 


74 


3.12 


3.51 


.39 


.001 


X 


c 




— — »» — 


~ *™ ~ ~ 


.17 






[D] 


X 


81 


3.42 


3.85 


.43 


.001 


c 


c 


201 


3.63 


4.13 


.50 


.001 






X 


89 


3.07 


4.33 


1.26 


.001 


X 


c 


60 


3.81 


4.2/ 


. HO 






[F] 


X 


82 


3.25 


3.63 


.38 


.001 


X 


c 


81 


3.49 


3.64 


. 15 


.05 




[6] 


X 


108 


4.32 


4.52 


.20 


.005 


X 


c 


36 


4.03 


3.86 


. 17 


NS 




[Hj 


X 


107 


3.15 


3.60 


.45 


.001 


X 




c 








.17 






flj 


X 


49 


4.43 


4.33 


.40 


.001 


X 


c 


38 


3.82 


4.03 


.21 


.05 




[J] 


X 


22 


3.72 


3.45 


- .27 


NS 


c 


L 


~ — ~ - 





■* — " 


. 17 


— ™ ■• — 




[K] 


X 


89 


3.80 


3.84 


.04 


NS 


c 


c 


49 


3.72 


4.07 


.35 


.005 




[L] 


X 


58 


4.12 


4.20 


.08 


NS 


c 


c 


97 


3.75 


4.09 


.34 


.001 




CM] 


X 


67 


4.12 


4.06 


- .06 


NS 


c 


c 


35 


3.89 


3.97 


.OR 


NS 




[N] 


X 


98 


3 M 


3 . 90 


. "*i< 


. un r > 


X 


c 


17 


3.76 


L7fi 


. no 


fi'. 






x 


67 


\ .23 


3 H' 




.00 1 


V 
A 




c 


141 


3.8? 


3.86 


.0-1 


NS 






X 


59 


3.64 


4.16 


.52 


.001 


X 




c 


107 


3.32 


3.55 


^ "i 
. L J 


.005 




MEANS 


X 


74 






.29 ' 




n=x 




c 


83 






.17 




5=C 


X = F.xp»." 


r inu-nt o 1 






29 









q r « Control 

ERIC 



Table 6 

Writing Assessment Gains By School 
(Conventions Score) 





ui uup 


N 


Pro 




P -a i n 

udi n 




[A] 


X 


84 


3.99 


4.02 


.03 


NS 




r 

V- 


I DO 


a r\A 




no 




[B] 


X 


47 


3.86 


4.06 


.20 


.01 




r 




^ QQ 






MC 
11 J 


[C3 


X 


74 


3.47 


3.68 


.21 


.005 




r 








1 A 
• 14 




[D] 


X 


81 


3.60 


3.87 


.27 


.001 




r 




1 7? 
J » 1 c 


a nft 

4 • UO 


. oO 




[E] 


X 


89 


3.66 


4.18 


.52 


.001 




c 


60 


3.95 


4.11 


.16 


.05 


[F] 


X 


82 


3.40 


3.73 


.33 


.001 




r 


ftl 




J • I c 


1 Q 

• iy 


• UUD 


[G] 


X 


108 


4.15 


4.42 


.27 


.001 




r 


3D 


1 QQ 


3 .y,> 


OA 




[H] 


X 


107 


3.39 


3.68 


.29 


.001 




r 








1 A 
.14 




[I] 


X 


49 


4.38 


4.69 


.31 


.005 




r 


oo 




A 1ft 
4.10 


• U4 


No 


[J] 


X 


22 


3.61 


3.70 


.09 


NS 




r 








1 /l 
• 14 




[KJ 


X 


89 


3.80 


3.84 


.04 


NS 




r 


AO 

**y 


j.t>y 


O 07 

j .y/ 


.28 


AAC 

.005 


[L] 


X 


58 


4.04 


4.18 


.14 


NS 




r 


Q7 

y / 


7 7f\ 


5 .y/ 




A A 1 

.001 


[Mj 


X 


67 


4.19 


4.25 


.06 


NS 




r 




7 Q1 

j .y 1 


/I HQ 
4 .Uo 


1 7 


MC 

NS 




x 




^ 7ft 


a no 


. 1 I 


fin i 




c 


17 


3.79 


3.97 


.IK 


NS 


[0] 


X 


67 


3.40 


3.6? 


.22 


.005 




c 


141 


3.92 


3.98 


.06 


NS 


[p] 


X 


59 


3.90 


4.31 


.41 


.001 




c 


107 


3.47 


3.54 


.07 


NS 


MEANS 


X 

c 


74 
83 






.23 
.14 





Gr. Gain 



X = Kxpor i mental 
C = Control 
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X 

c 

X 
X 
X 
X 
X 

c 
c 
c 

c 



5=C 



25 



control groups on organization, development, conventions and total scores 
show statistically significant differences favoring the experimental 
program. Only the focus score failed to show a significant difference . 

An examination of Tables 2-6 shows similarly favorable results at the 
individual school level. In these tables, experimental and control groups 
for each individual school are shown. For each group, the number of 
subjects (N) with complete pre- and posttest records is shown. Axso shown 
are the mean pre- and posttest score and the gain (Gain) or difference 
score. The results of a correlated t-test (t-sig) comparing pre- and 
posttest scores are also represented in terms of the significance level 
reached. Finally, the group with the greater gain (Gr. Gain) is identified 
in the final column of the tables. For each score type listed in these 
tables, the number of cases where experimental school groups gained more 
than the control groups exceeded by a wide margin the number of cases in 
which the controls outgained the experimental s . 

Thus, the weight of evidence indicated by the findings supports the 
contention that the computer writing program does indeed serve to enhance 
the writing skills of participating students. 

D id students get more enjoyment of writing as a result of participation 
in the project? Student enjoyment of writing was assessed using a single 
question on the student attitude survey administered to both experimental 
and control groups on a pre- and posttest basis. The question c ntent and 
analysis results are given below. 

Quest ion : Do you enjoy working on writing assignments? 



1. Almost Always 

Z. Often 

3. Sometimes 

4. Rarely 
5>. Never 
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Group N Pre Post Gain t-sig Gr. Gain 

X 1119 2.90 2.67 - .23 .001 X 

C 1099 3.07 3.08 + .01 NS 

Since the response scale for the item is arranged from "1 « Almost 
Always" to "5 * Never, " movement from a higher pretest mean to a lower 
posttest mean is interpreted to be a more favorable response {i.e., a gain 
in enjoyment)* As can be seen by the results, only the experimental group 
gained in its enjoyment of writing from the beginning to the end of the 
academic year, and this gain was substantial as well as being statistically 
significant. On-site observations and discussions with teachers and 
students at the experimental group sites also confirmed that experimental 
students gained in their enjoyment of the writing process. 

Did students enjoys using the computer for learning and practicing 
writing skllls7 Student enjoyment of use of the computer for learning and 
practicing writing skills was also assessed using a single item on the 
student survey. The survey question and the results for experimental and 
control groups are presented below. 

Question? Do you enjoy using a computer for writing? 



1. 


Almost Always 








2. 


Often 








3. 


Somet imes 








4. 


Rarely 








5, 


Never 








Group 


N Pre 


Post 


Gain t-sig 


Gr. Gain 


X 


1113 2.39 


1 .70 


- .69 .001 


' X 


c 


1095 2.78 


2.66 


- .12 .01 




The 


response scale ranges 


from a "1 


- Almost Always " to 


a *5 « Never", 


and thus 


changes in the direction of more 


enjoyment from pre- 


to posttest 



would result in a negative difference score. The magnitude of the negative 
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difference score would indicate the extent of the more favorable change 
experienced. 

As can be seen by the above results, the experimental program group 
shows a gain in enjoyment of computer use for learning and practicing 
writing skills. The gain is also more than five times that of the control 
group. This is to be expected since the exposure of the control group 
students to computers for use in writing on a formal basis was to have been 
of a limited nature . 

It is clear, then, that the experimental program group enjoyed using 
the computers. On-site observations and discussions with teachers and 
students served to confirm this finding as well. 

Conclusions and Implications 

Based on the evaluation study of the computer writing program, several 
conclusions can be drawn. 

• In general, adequate orientation and training were provided to the 
participating schools as well as help in selecting and acquiring 
computer hardware and software. 

• Most program sites started on time and implemented the program as 
planned. 

• Despite all efforts, a few of the schools did not take advantage of 
the special training opportunities and technical assistance 
services and, in some cases, struggled somewhat during 
implementation as a consequence. 

• It appears that on-site visitation and monitoring of the first year 
of implementation of a computer writing program is essential to its 
success and continuation. 

• Student writing skills are enhanced through use of computers within 
the context of a process approach to writing. 

• Students enjoy writing more when using computers in conjunction 
with their normal writing instruction program. 

• Students enjoy using the computer for learning and practicing their 
writing skills. 
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